This is a notebook to keep logs on running the training procedure for the first time, with some simple preprocessing options. Have already sourced the Python 2 virtual environment set up according to instructions in the README from the tools repo and sourced the script to set environment variables:
source start_scriptChecking that this has worked by importing Theano and looking at its config:
In [1]:
import theano
print(theano.config.device)
gpu0
Using gpu device 0: Tesla K40c
Looks like it's set up to use the GPU ok.
Running the code should then be as simple as pointing at the right run config json file. Using the following file:
In [2]:
cd ..
/afs/inf.ed.ac.uk/user/s08/s0805516/repos/neukrill-net-work
In [3]:
!cat run_settings/first_pylearn2_run.json
{
"model type":"pylearn2",
"yaml file": "default.yaml",
"preprocessing":{"resize":[48,48]},
"final_shape":[48,48],
"augmentation_factor":1,
"train_split": 0.8
}
Doing practically nothing except resizing to quite a small size. A major problem at this point is likely to be the lack of image normalization.
The YAML file specified is what controls Pylearn2's operation. The file we're using has been cannabalised from the MNIST example bundled with Pylearn2. It's in the repo and looks like this:
In [4]:
!cat yaml_templates/default.yaml
!obj:pylearn2.train.Train {
dataset: &train !obj:neukrill_net.dense_dataset.DensePNGDataset {
settings_path: %(settings_path)s,
run_settings: %(run_settings_path)s,
training_set_mode: "train"
},
model: !obj:pylearn2.models.mlp.MLP {
batch_size: 128,
input_space: !obj:pylearn2.space.Conv2DSpace {
shape: %(final_shape)s,
num_channels: 1,
axes: ['c', 0, 1, 'b'],
},
layers: [
!obj:pylearn2.models.maxout.MaxoutConvC01B {
layer_name: 'h0',
pad: 0,
num_channels: 48,
num_pieces: 2,
kernel_shape: [8, 8],
pool_shape: [4, 4],
pool_stride: [2, 2],
irange: .005,
max_kernel_norm: .9,
},
!obj:pylearn2.models.maxout.MaxoutConvC01B {
layer_name: 'h1',
pad: 3,
num_channels: 48,
num_pieces: 2,
kernel_shape: [8, 8],
pool_shape: [4, 4],
pool_stride: [2, 2],
irange: .005,
max_kernel_norm: 1.9365,
},
!obj:pylearn2.models.maxout.MaxoutConvC01B {
pad: 3,
layer_name: 'h2',
num_channels: 24,
num_pieces: 4,
kernel_shape: [5, 5],
pool_shape: [2, 2],
pool_stride: [2, 2],
irange: .005,
max_kernel_norm: 1.9365,
},
!obj:pylearn2.models.mlp.Softmax {
max_col_norm: 1.9365,
layer_name: 'y',
n_classes: %(n_classes)s,
irange: .005
}
],
},
algorithm: !obj:pylearn2.training_algorithms.sgd.SGD {
learning_rate: .05,
learning_rule: !obj:pylearn2.training_algorithms.learning_rule.Momentum {
init_momentum: 0.5,
},
train_iteration_mode: 'even_shuffled_sequential',
monitor_iteration_mode: 'even_sequential',
monitoring_dataset:
{
'valid' : !obj:neukrill_net.dense_dataset.DensePNGDataset {
settings_path: %(settings_path)s,
run_settings: %(run_settings_path)s,
training_set_mode: "validation"
},
'test' : !obj:neukrill_net.dense_dataset.DensePNGDataset {
settings_path: %(settings_path)s,
run_settings: %(run_settings_path)s,
training_set_mode: "test"
}
},
cost: !obj:pylearn2.costs.mlp.dropout.Dropout {
input_include_probs: { 'h0' : .8 },
input_scales: { 'h0': 1. }
},
termination_criterion: !obj:pylearn2.termination_criteria.And {
criteria: [
!obj:pylearn2.termination_criteria.MonitorBased {
channel_name: "valid_y_misclass",
prop_decrease: 0.50,
N: 10
},
!obj:pylearn2.termination_criteria.EpochCounter {
max_epochs: 500
},
]
},
},
extensions:
[ !obj:pylearn2.train_extensions.best_params.MonitorBasedSaveBest {
channel_name: 'valid_y_misclass',
save_path: %(save_path)s
}, !obj:pylearn2.training_algorithms.learning_rule.MomentumAdjustor {
start: 1,
saturate: 10,
final_momentum: .99
}
]
}
All we need to do is point our train.py script (don't get this confused with either the train module from Pylearn2 or its own train.py script (unimaginative naming problems...)) at the above json:
In [ ]:
%pdb
Automatic pdb calling has been turned ON
In [ ]:
%run train.py run_settings/first_pylearn2_run.json
Input shape: (48, 48)
Detector space: (41, 41)
Output space: (20, 20)
Input shape: (20, 20)
Detector space: (19, 19)
Output space: (9, 9)
Input shape: (9, 9)
Detector space: (11, 11)
Output space: (6, 6)
Parameter and initial learning rate summary:
h0_W: 0.0500000007451
h0_b: 0.0500000007451
h1_W: 0.0500000007451
h1_b: 0.0500000007451
h2_W: 0.0500000007451
h2_b: 0.0500000007451
softmax_b: 0.0500000007451
softmax_W: 0.0500000007451
Compiling sgd_update...
/afs/inf.ed.ac.uk/user/s08/s0805516/repos/pylearn2/pylearn2/utils/iteration.py:783: UserWarning: dataset is using the old iterator interface which is deprecated and will become officially unsupported as of July 28, 2015. The dataset should implement a `get` method respecting the new interface.
warnings.warn("dataset is using the old iterator interface which "
Compiling sgd_update done. Time elapsed: 3.973367 seconds
compiling begin_record_entry...
compiling begin_record_entry done. Time elapsed: 0.544542 seconds
Monitored channels:
learning_rate
momentum
test_h0_kernel_norms_max
test_h0_kernel_norms_mean
test_h0_kernel_norms_min
test_h0_max_x.max_u
test_h0_max_x.mean_u
test_h0_max_x.min_u
test_h0_mean_x.max_u
test_h0_mean_x.mean_u
test_h0_mean_x.min_u
test_h0_min_x.max_u
test_h0_min_x.mean_u
test_h0_min_x.min_u
test_h0_range_x.max_u
test_h0_range_x.mean_u
test_h0_range_x.min_u
test_h1_kernel_norms_max
test_h1_kernel_norms_mean
test_h1_kernel_norms_min
test_h1_max_x.max_u
test_h1_max_x.mean_u
test_h1_max_x.min_u
test_h1_mean_x.max_u
test_h1_mean_x.mean_u
test_h1_mean_x.min_u
test_h1_min_x.max_u
test_h1_min_x.mean_u
test_h1_min_x.min_u
test_h1_range_x.max_u
test_h1_range_x.mean_u
test_h1_range_x.min_u
test_h2_kernel_norms_max
test_h2_kernel_norms_mean
test_h2_kernel_norms_min
test_h2_max_x.max_u
test_h2_max_x.mean_u
test_h2_max_x.min_u
test_h2_mean_x.max_u
test_h2_mean_x.mean_u
test_h2_mean_x.min_u
test_h2_min_x.max_u
test_h2_min_x.mean_u
test_h2_min_x.min_u
test_h2_range_x.max_u
test_h2_range_x.mean_u
test_h2_range_x.min_u
test_objective
test_y_col_norms_max
test_y_col_norms_mean
test_y_col_norms_min
test_y_max_max_class
test_y_mean_max_class
test_y_min_max_class
test_y_misclass
test_y_nll
test_y_row_norms_max
test_y_row_norms_mean
test_y_row_norms_min
total_seconds_last_epoch
training_seconds_this_epoch
valid_h0_kernel_norms_max
valid_h0_kernel_norms_mean
valid_h0_kernel_norms_min
valid_h0_max_x.max_u
valid_h0_max_x.mean_u
valid_h0_max_x.min_u
valid_h0_mean_x.max_u
valid_h0_mean_x.mean_u
valid_h0_mean_x.min_u
valid_h0_min_x.max_u
valid_h0_min_x.mean_u
valid_h0_min_x.min_u
valid_h0_range_x.max_u
valid_h0_range_x.mean_u
valid_h0_range_x.min_u
valid_h1_kernel_norms_max
valid_h1_kernel_norms_mean
valid_h1_kernel_norms_min
valid_h1_max_x.max_u
valid_h1_max_x.mean_u
valid_h1_max_x.min_u
valid_h1_mean_x.max_u
valid_h1_mean_x.mean_u
valid_h1_mean_x.min_u
valid_h1_min_x.max_u
valid_h1_min_x.mean_u
valid_h1_min_x.min_u
valid_h1_range_x.max_u
valid_h1_range_x.mean_u
valid_h1_range_x.min_u
valid_h2_kernel_norms_max
valid_h2_kernel_norms_mean
valid_h2_kernel_norms_min
valid_h2_max_x.max_u
valid_h2_max_x.mean_u
valid_h2_max_x.min_u
valid_h2_mean_x.max_u
valid_h2_mean_x.mean_u
valid_h2_mean_x.min_u
valid_h2_min_x.max_u
valid_h2_min_x.mean_u
valid_h2_min_x.min_u
valid_h2_range_x.max_u
valid_h2_range_x.mean_u
valid_h2_range_x.min_u
valid_objective
valid_y_col_norms_max
valid_y_col_norms_mean
valid_y_col_norms_min
valid_y_max_max_class
valid_y_mean_max_class
valid_y_min_max_class
valid_y_misclass
valid_y_nll
valid_y_row_norms_max
valid_y_row_norms_mean
valid_y_row_norms_min
Compiling accum...
graph size: 387
graph size: 383
Compiling accum done. Time elapsed: 13.744146 seconds
Monitoring step:
Epochs seen: 0
Batches seen: 0
Examples seen: 0
learning_rate: 0.0500000156462
momentum: 0.500000059605
test_h0_kernel_norms_max: 0.025620367378
test_h0_kernel_norms_mean: 0.0229787509888
test_h0_kernel_norms_min: 0.0195152591914
test_h0_max_x.max_u: 0.0695547536016
test_h0_max_x.mean_u: 0.0313058644533
test_h0_max_x.min_u: 0.00327901006676
test_h0_mean_x.max_u: 0.0611211955547
test_h0_mean_x.mean_u: 0.0161500163376
test_h0_mean_x.min_u: -0.0205041710287
test_h0_min_x.max_u: 0.0150190703571
test_h0_min_x.mean_u: 0.000894624390639
test_h0_min_x.min_u: -0.0246901288629
test_h0_range_x.max_u: 0.0549388863146
test_h0_range_x.mean_u: 0.030411247164
test_h0_range_x.min_u: 0.0161684025079
test_h1_kernel_norms_max: 0.163002476096
test_h1_kernel_norms_mean: 0.160210266709
test_h1_kernel_norms_min: 0.157817587256
test_h1_max_x.max_u: 0.0124799013138
test_h1_max_x.mean_u: 0.00550353666767
test_h1_max_x.min_u: -0.00122586847283
test_h1_mean_x.max_u: 0.0103779649362
test_h1_mean_x.mean_u: 0.00287891621701
test_h1_mean_x.min_u: -0.00506942579523
test_h1_min_x.max_u: 0.00670026056468
test_h1_min_x.mean_u: 0.000480789778521
test_h1_min_x.min_u: -0.00764158414677
test_h1_range_x.max_u: 0.00772560434416
test_h1_range_x.mean_u: 0.00502274744213
test_h1_range_x.min_u: 0.00309842941351
test_h2_kernel_norms_max: 0.102913320065
test_h2_kernel_norms_mean: 0.0999667793512
test_h2_kernel_norms_min: 0.0971000939608
test_h2_max_x.max_u: 0.00137931655627
test_h2_max_x.mean_u: 0.000808427110314
test_h2_max_x.min_u: 0.000285953807179
test_h2_mean_x.max_u: 0.000732974614948
test_h2_mean_x.mean_u: 0.000421099801315
test_h2_mean_x.min_u: 5.88356633671e-05
test_h2_min_x.max_u: 0.000293726770906
test_h2_min_x.mean_u: 5.37531268492e-05
test_h2_min_x.min_u: -0.000307372451061
test_h2_range_x.max_u: 0.00121322902851
test_h2_range_x.mean_u: 0.000754674081691
test_h2_range_x.min_u: 0.00044881858048
test_objective: 4.79580688477
test_y_col_norms_max: 0.0886595323682
test_y_col_norms_mean: 0.0852382704616
test_y_col_norms_min: 0.0822916701436
test_y_max_max_class: 0.00826539006084
test_y_mean_max_class: 0.00826532393694
test_y_min_max_class: 0.00826526526362
test_y_misclass: 0.999023616314
test_y_nll: 4.79579114914
test_y_row_norms_max: 0.0359390377998
test_y_row_norms_mean: 0.0318758152425
test_y_row_norms_min: 0.0280178301036
total_seconds_last_epoch: 0.0
training_seconds_this_epoch: 0.0
valid_h0_kernel_norms_max: 0.0256203617901
valid_h0_kernel_norms_mean: 0.0229787528515
valid_h0_kernel_norms_min: 0.0195152573287
valid_h0_max_x.max_u: 0.0694105625153
valid_h0_max_x.mean_u: 0.0311431288719
valid_h0_max_x.min_u: 0.00300892512314
valid_h0_mean_x.max_u: 0.0612021870911
valid_h0_mean_x.mean_u: 0.0161678306758
valid_h0_mean_x.min_u: -0.0205619186163
valid_h0_min_x.max_u: 0.0139615628868
valid_h0_min_x.mean_u: 0.000747101788875
valid_h0_min_x.min_u: -0.0246833171695
valid_h0_range_x.max_u: 0.0559068731964
valid_h0_range_x.mean_u: 0.0303960237652
valid_h0_range_x.min_u: 0.0165484864265
valid_h1_kernel_norms_max: 0.163002505898
valid_h1_kernel_norms_mean: 0.160210311413
valid_h1_kernel_norms_min: 0.157817557454
valid_h1_max_x.max_u: 0.0124876089394
valid_h1_max_x.mean_u: 0.00549738807604
valid_h1_max_x.min_u: -0.0010639752727
valid_h1_mean_x.max_u: 0.0103793274611
valid_h1_mean_x.mean_u: 0.00287629244849
valid_h1_mean_x.min_u: -0.00508263986558
valid_h1_min_x.max_u: 0.00675621815026
valid_h1_min_x.mean_u: 0.000497908622492
valid_h1_min_x.min_u: -0.00762424245477
valid_h1_range_x.max_u: 0.00757502578199
valid_h1_range_x.mean_u: 0.00499947974458
valid_h1_range_x.min_u: 0.00308659533039
valid_h2_kernel_norms_max: 0.102913297713
valid_h2_kernel_norms_mean: 0.0999667495489
valid_h2_kernel_norms_min: 0.0971001237631
valid_h2_max_x.max_u: 0.00137885136064
valid_h2_max_x.mean_u: 0.000807397649623
valid_h2_max_x.min_u: 0.000289376825094
valid_h2_mean_x.max_u: 0.000732935382985
valid_h2_mean_x.mean_u: 0.000421148986788
valid_h2_mean_x.min_u: 5.85719062656e-05
valid_h2_min_x.max_u: 0.000297168618999
valid_h2_min_x.mean_u: 5.5023621826e-05
valid_h2_min_x.min_u: -0.000307764275931
valid_h2_range_x.max_u: 0.00120884552598
valid_h2_range_x.mean_u: 0.000752374064177
valid_h2_range_x.min_u: 0.000442488206318
valid_objective: 4.79580688477
valid_y_col_norms_max: 0.0886595696211
valid_y_col_norms_mean: 0.0852382481098
valid_y_col_norms_min: 0.0822916552424
valid_y_max_max_class: 0.00826539844275
valid_y_mean_max_class: 0.00826532579958
valid_y_min_max_class: 0.00826526526362
valid_y_misclass: 0.998980820179
valid_y_nll: 4.79579162598
valid_y_row_norms_max: 0.0359390377998
valid_y_row_norms_mean: 0.0318758152425
valid_y_row_norms_min: 0.0280178263783
Saving to /disk/scratch/neuroglycerin/models/first_pylearn2_run.pkl...
Saving to /disk/scratch/neuroglycerin/models/first_pylearn2_run.pkl done. Time elapsed: 0.402772 seconds
Time this epoch: 42.365288 seconds
Monitoring step:
Epochs seen: 1
Batches seen: 189
Examples seen: 24192
learning_rate: 0.0500000156462
momentum: 0.500000059605
test_h0_kernel_norms_max: 0.119630277157
test_h0_kernel_norms_mean: 0.0446750447154
test_h0_kernel_norms_min: 0.0240400601178
test_h0_max_x.max_u: 0.0877726972103
test_h0_max_x.mean_u: -0.0394509211183
test_h0_max_x.min_u: -0.108581222594
test_h0_mean_x.max_u: 0.0777599066496
test_h0_mean_x.mean_u: -0.222826212645
test_h0_mean_x.min_u: -0.584578573704
test_h0_min_x.max_u: 0.0195010378957
test_h0_min_x.mean_u: -0.248172312975
test_h0_min_x.min_u: -0.646384298801
test_h0_range_x.max_u: 0.537803173065
test_h0_range_x.mean_u: 0.208721384406
test_h0_range_x.min_u: 0.0424815453589
test_h1_kernel_norms_max: 0.178259134293
test_h1_kernel_norms_mean: 0.164288386703
test_h1_kernel_norms_min: 0.157976344228
test_h1_max_x.max_u: 0.728937506676
test_h1_max_x.mean_u: 0.406539261341
test_h1_max_x.min_u: 0.155406236649
test_h1_mean_x.max_u: 0.575318336487
test_h1_mean_x.mean_u: 0.320599079132
test_h1_mean_x.min_u: 0.115812256932
test_h1_min_x.max_u: 0.190403625369
test_h1_min_x.mean_u: 0.0893200114369
test_h1_min_x.min_u: 0.00710412999615
test_h1_range_x.max_u: 0.565589189529
test_h1_range_x.mean_u: 0.317219287157
test_h1_range_x.min_u: 0.112690351903
test_h2_kernel_norms_max: 0.173589587212
test_h2_kernel_norms_mean: 0.108563363552
test_h2_kernel_norms_min: 0.0981491953135
test_h2_max_x.max_u: 1.1296530962
test_h2_max_x.mean_u: 0.534217238426
test_h2_max_x.min_u: 0.111819863319
test_h2_mean_x.max_u: 0.644700169563
test_h2_mean_x.mean_u: 0.302310168743
test_h2_mean_x.min_u: 0.054583966732
test_h2_min_x.max_u: 0.0786329656839
test_h2_min_x.mean_u: 0.0263672303408
test_h2_min_x.min_u: -0.0340515673161
test_h2_range_x.max_u: 1.09922862053
test_h2_range_x.mean_u: 0.507849991322
test_h2_range_x.min_u: 0.128865197301
test_objective: 4.17824888229
test_y_col_norms_max: 0.514889061451
test_y_col_norms_mean: 0.162969008088
test_y_col_norms_min: 0.101006679237
test_y_max_max_class: 0.0681242123246
test_y_mean_max_class: 0.0506275594234
test_y_min_max_class: 0.041659720242
test_y_misclass: 0.916015684605
test_y_nll: 4.17635774612
test_y_row_norms_max: 0.164149001241
test_y_row_norms_mean: 0.0595046244562
test_y_row_norms_min: 0.028547193855
total_seconds_last_epoch: 0.0
training_seconds_this_epoch: 42.3652915955
valid_h0_kernel_norms_max: 0.119630292058
valid_h0_kernel_norms_mean: 0.0446750298142
valid_h0_kernel_norms_min: 0.0240400545299
valid_h0_max_x.max_u: 0.0871249958873
valid_h0_max_x.mean_u: -0.0393340885639
valid_h0_max_x.min_u: -0.108536668122
valid_h0_mean_x.max_u: 0.0778748095036
valid_h0_mean_x.mean_u: -0.223461583257
valid_h0_mean_x.min_u: -0.586237847805
valid_h0_min_x.max_u: 0.0187591761351
valid_h0_min_x.mean_u: -0.24818123877
valid_h0_min_x.min_u: -0.646384358406
valid_h0_range_x.max_u: 0.537847697735
valid_h0_range_x.mean_u: 0.208847150207
valid_h0_range_x.min_u: 0.0422600880265
valid_h1_kernel_norms_max: 0.17825910449
valid_h1_kernel_norms_mean: 0.164288416505
valid_h1_kernel_norms_min: 0.157976314425
valid_h1_max_x.max_u: 0.728963315487
valid_h1_max_x.mean_u: 0.406164675951
valid_h1_max_x.min_u: 0.155447885394
valid_h1_mean_x.max_u: 0.576275587082
valid_h1_mean_x.mean_u: 0.320993304253
valid_h1_mean_x.min_u: 0.115926712751
valid_h1_min_x.max_u: 0.186443910003
valid_h1_min_x.mean_u: 0.0852651894093
valid_h1_min_x.min_u: 0.00356428069063
valid_h1_range_x.max_u: 0.568383038044
valid_h1_range_x.mean_u: 0.320899426937
valid_h1_range_x.min_u: 0.112883076072
valid_h2_kernel_norms_max: 0.173589587212
valid_h2_kernel_norms_mean: 0.108563423157
valid_h2_kernel_norms_min: 0.0981492251158
valid_h2_max_x.max_u: 1.12555992603
valid_h2_max_x.mean_u: 0.531945824623
valid_h2_max_x.min_u: 0.111693903804
valid_h2_mean_x.max_u: 0.645218133926
valid_h2_mean_x.mean_u: 0.302555888891
valid_h2_mean_x.min_u: 0.0546179562807
valid_h2_min_x.max_u: 0.0830293595791
valid_h2_min_x.mean_u: 0.0270454064012
valid_h2_min_x.min_u: -0.0341226831079
valid_h2_range_x.max_u: 1.09422528744
valid_h2_range_x.mean_u: 0.504900336266
valid_h2_range_x.min_u: 0.128659635782
valid_objective: 4.14758968353
valid_y_col_norms_max: 0.514889240265
valid_y_col_norms_mean: 0.162969052792
valid_y_col_norms_min: 0.101006694138
valid_y_max_max_class: 0.0662094578147
valid_y_mean_max_class: 0.0503454096615
valid_y_min_max_class: 0.0423163473606
valid_y_misclass: 0.912363946438
valid_y_nll: 4.13866329193
valid_y_row_norms_max: 0.164148956537
valid_y_row_norms_mean: 0.0595046319067
valid_y_row_norms_min: 0.0285471752286
Saving to /disk/scratch/neuroglycerin/models/first_pylearn2_run.pkl...
Saving to /disk/scratch/neuroglycerin/models/first_pylearn2_run.pkl done. Time elapsed: 0.272643 seconds
Time this epoch: 42.344463 seconds
Monitoring step:
Epochs seen: 2
Batches seen: 378
Examples seen: 48384
learning_rate: 0.0500000156462
momentum: 0.554444611073
test_h0_kernel_norms_max: 0.209845691919
test_h0_kernel_norms_mean: 0.0957242920995
test_h0_kernel_norms_min: 0.0244844965637
test_h0_max_x.max_u: 0.200352042913
test_h0_max_x.mean_u: 0.00634754262865
test_h0_max_x.min_u: -0.0850978121161
test_h0_mean_x.max_u: 0.117207296193
test_h0_mean_x.mean_u: -0.396451234818
test_h0_mean_x.min_u: -0.804564297199
test_h0_min_x.max_u: 0.0272604301572
test_h0_min_x.mean_u: -0.46067199111
test_h0_min_x.min_u: -0.910667657852
test_h0_range_x.max_u: 0.873636364937
test_h0_range_x.mean_u: 0.467019468546
test_h0_range_x.min_u: 0.0543017201126
test_h1_kernel_norms_max: 0.501127421856
test_h1_kernel_norms_mean: 0.22133846581
test_h1_kernel_norms_min: 0.162283405662
test_h1_max_x.max_u: 2.85028910637
test_h1_max_x.mean_u: 1.04021894932
test_h1_max_x.min_u: -0.0126008475199
test_h1_mean_x.max_u: 0.865136742592
test_h1_mean_x.mean_u: -0.116089887917
test_h1_mean_x.min_u: -1.00601768494
test_h1_min_x.max_u: -0.113007441163
test_h1_min_x.mean_u: -1.01708626747
test_h1_min_x.min_u: -2.24564099312
test_h1_range_x.max_u: 4.86607980728
test_h1_range_x.mean_u: 2.057305336
test_h1_range_x.min_u: 0.257792323828
test_h2_kernel_norms_max: 0.436911284924
test_h2_kernel_norms_mean: 0.170230612159
test_h2_kernel_norms_min: 0.0996099039912
test_h2_max_x.max_u: 5.733086586
test_h2_max_x.mean_u: 3.39873337746
test_h2_max_x.min_u: 0.718877494335
test_h2_mean_x.max_u: 0.780628621578
test_h2_mean_x.mean_u: 0.427675038576
test_h2_mean_x.min_u: 0.0342684164643
test_h2_min_x.max_u: -0.124309390783
test_h2_min_x.mean_u: -1.29148459435
test_h2_min_x.min_u: -2.32623887062
test_h2_range_x.max_u: 7.97482538223
test_h2_range_x.mean_u: 4.6902179718
test_h2_range_x.min_u: 0.883309841156
test_objective: 3.49535536766
test_y_col_norms_max: 0.831780970097
test_y_col_norms_mean: 0.290325641632
test_y_col_norms_min: 0.148055061698
test_y_max_max_class: 0.892422378063
test_y_mean_max_class: 0.374676644802
test_y_min_max_class: 0.0861578956246
test_y_misclass: 0.7919921875
test_y_nll: 3.54511523247
test_y_row_norms_max: 0.328452318907
test_y_row_norms_mean: 0.103720292449
test_y_row_norms_min: 0.0298915319145
total_seconds_last_epoch: 47.7596664429
training_seconds_this_epoch: 42.344455719
valid_h0_kernel_norms_max: 0.209845721722
valid_h0_kernel_norms_mean: 0.0957242697477
valid_h0_kernel_norms_min: 0.0244844835252
valid_h0_max_x.max_u: 0.199420377612
valid_h0_max_x.mean_u: 0.00478140218183
valid_h0_max_x.min_u: -0.0860378965735
valid_h0_mean_x.max_u: 0.117339596152
valid_h0_mean_x.mean_u: -0.397686719894
valid_h0_mean_x.min_u: -0.806932985783
valid_h0_min_x.max_u: 0.024752497673
valid_h0_min_x.mean_u: -0.46079492569
valid_h0_min_x.min_u: -0.910667419434
valid_h0_range_x.max_u: 0.872015595436
valid_h0_range_x.mean_u: 0.465576380491
valid_h0_range_x.min_u: 0.0554380491376
valid_h1_kernel_norms_max: 0.501127481461
valid_h1_kernel_norms_mean: 0.221338480711
valid_h1_kernel_norms_min: 0.162283465266
valid_h1_max_x.max_u: 2.73578166962
valid_h1_max_x.mean_u: 1.01325666904
valid_h1_max_x.min_u: -0.015496565029
valid_h1_mean_x.max_u: 0.861131668091
valid_h1_mean_x.mean_u: -0.123113930225
valid_h1_mean_x.min_u: -1.01675236225
valid_h1_min_x.max_u: -0.128132730722
valid_h1_min_x.mean_u: -1.01643753052
valid_h1_min_x.min_u: -2.2457382679
valid_h1_range_x.max_u: 4.74751996994
valid_h1_range_x.mean_u: 2.02969408035
valid_h1_range_x.min_u: 0.255120128393
valid_h2_kernel_norms_max: 0.436911404133
valid_h2_kernel_norms_mean: 0.17023062706
valid_h2_kernel_norms_min: 0.0996099114418
valid_h2_max_x.max_u: 5.67662525177
valid_h2_max_x.mean_u: 3.3808825016
valid_h2_max_x.min_u: 0.705309808254
valid_h2_mean_x.max_u: 0.783128023148
valid_h2_mean_x.mean_u: 0.427880227566
valid_h2_mean_x.min_u: 0.0258177462965
valid_h2_min_x.max_u: -0.119625985622
valid_h2_min_x.mean_u: -1.29079389572
valid_h2_min_x.min_u: -2.28750252724
valid_h2_range_x.max_u: 7.8768157959
valid_h2_range_x.mean_u: 4.67167615891
valid_h2_range_x.min_u: 0.86976402998
valid_objective: 3.4253821373
valid_y_col_norms_max: 0.831781148911
valid_y_col_norms_mean: 0.29032561183
valid_y_col_norms_min: 0.148055061698
valid_y_max_max_class: 0.8895637393
valid_y_mean_max_class: 0.371127307415
valid_y_min_max_class: 0.0873955860734
valid_y_misclass: 0.783967316151
valid_y_nll: 3.46552085876
valid_y_row_norms_max: 0.328452348709
valid_y_row_norms_mean: 0.103720247746
valid_y_row_norms_min: 0.0298915486783
Saving to /disk/scratch/neuroglycerin/models/first_pylearn2_run.pkl...
Saving to /disk/scratch/neuroglycerin/models/first_pylearn2_run.pkl done. Time elapsed: 0.272255 seconds
Time this epoch: 42.376332 seconds
Monitoring step:
Epochs seen: 3
Batches seen: 567
Examples seen: 72576
learning_rate: 0.0500000156462
momentum: 0.608888745308
test_h0_kernel_norms_max: 0.359491884708
test_h0_kernel_norms_mean: 0.134614363313
test_h0_kernel_norms_min: 0.024905398488
test_h0_max_x.max_u: 0.423915982246
test_h0_max_x.mean_u: 0.0890484005213
test_h0_max_x.min_u: -0.0862857550383
test_h0_mean_x.max_u: 0.121149219573
test_h0_mean_x.mean_u: -0.404546886683
test_h0_mean_x.min_u: -0.728514194489
test_h0_min_x.max_u: -0.0107742007822
test_h0_min_x.mean_u: -0.496399521828
test_h0_min_x.min_u: -0.863461852074
test_h0_range_x.max_u: 1.18968904018
test_h0_range_x.mean_u: 0.585447967052
test_h0_range_x.min_u: 0.108081080019
test_h1_kernel_norms_max: 0.617425322533
test_h1_kernel_norms_mean: 0.278276771307
test_h1_kernel_norms_min: 0.168387040496
test_h1_max_x.max_u: 3.59739923477
test_h1_max_x.mean_u: 1.67295086384
test_h1_max_x.min_u: 0.00620114803314
test_h1_mean_x.max_u: 1.05822396278
test_h1_mean_x.mean_u: -0.0799116566777
test_h1_mean_x.min_u: -1.12907385826
test_h1_min_x.max_u: -0.076726295054
test_h1_min_x.mean_u: -1.39848601818
test_h1_min_x.min_u: -3.1450252533
test_h1_range_x.max_u: 5.78718328476
test_h1_range_x.mean_u: 3.07143688202
test_h1_range_x.min_u: 0.536547243595
test_h2_kernel_norms_max: 0.619913458824
test_h2_kernel_norms_mean: 0.248795241117
test_h2_kernel_norms_min: 0.100504353642
test_h2_max_x.max_u: 6.73342561722
test_h2_max_x.mean_u: 3.97779417038
test_h2_max_x.min_u: 1.6889333725
test_h2_mean_x.max_u: 0.673189640045
test_h2_mean_x.mean_u: 0.349344611168
test_h2_mean_x.min_u: -0.0794017761946
test_h2_min_x.max_u: -0.252630829811
test_h2_min_x.mean_u: -2.25534701347
test_h2_min_x.min_u: -3.62425899506
test_h2_range_x.max_u: 9.86682987213
test_h2_range_x.mean_u: 6.2331404686
test_h2_range_x.min_u: 1.9765740633
test_objective: 3.38402199745
test_y_col_norms_max: 1.02679502964
test_y_col_norms_mean: 0.416194468737
test_y_col_norms_min: 0.178296118975
test_y_max_max_class: 0.889362454414
test_y_mean_max_class: 0.445143610239
test_y_min_max_class: 0.121569037437
test_y_misclass: 0.767252624035
test_y_nll: 3.57071065903
test_y_row_norms_max: 0.423833578825
test_y_row_norms_mean: 0.151956483722
test_y_row_norms_min: 0.0329562090337
total_seconds_last_epoch: 47.7502403259
training_seconds_this_epoch: 42.3763389587
valid_h0_kernel_norms_max: 0.359492003918
valid_h0_kernel_norms_mean: 0.13461433351
valid_h0_kernel_norms_min: 0.0249053966254
valid_h0_max_x.max_u: 0.412712156773
valid_h0_max_x.mean_u: 0.0861949026585
valid_h0_max_x.min_u: -0.0869294703007
valid_h0_mean_x.max_u: 0.121100381017
valid_h0_mean_x.mean_u: -0.405968815088
valid_h0_mean_x.min_u: -0.731344103813
valid_h0_min_x.max_u: -0.011479396373
valid_h0_min_x.mean_u: -0.495568722486
valid_h0_min_x.min_u: -0.863550662994
valid_h0_range_x.max_u: 1.1776227951
valid_h0_range_x.mean_u: 0.581763625145
valid_h0_range_x.min_u: 0.108674034476
valid_h1_kernel_norms_max: 0.617425024509
valid_h1_kernel_norms_mean: 0.278276890516
valid_h1_kernel_norms_min: 0.168387085199
valid_h1_max_x.max_u: 3.46933794022
valid_h1_max_x.mean_u: 1.634745121
valid_h1_max_x.min_u: -0.0142243849114
valid_h1_mean_x.max_u: 1.05815958977
valid_h1_mean_x.mean_u: -0.08995449543
valid_h1_mean_x.min_u: -1.1476982832
valid_h1_min_x.max_u: -0.0819198489189
valid_h1_min_x.mean_u: -1.39712274075
valid_h1_min_x.min_u: -3.13098526001
valid_h1_range_x.max_u: 5.65823078156
valid_h1_range_x.mean_u: 3.03186798096
valid_h1_range_x.min_u: 0.529259443283
valid_h2_kernel_norms_max: 0.619913220406
valid_h2_kernel_norms_mean: 0.248795226216
valid_h2_kernel_norms_min: 0.100504390895
valid_h2_max_x.max_u: 6.68678617477
valid_h2_max_x.mean_u: 3.95872044563
valid_h2_max_x.min_u: 1.6506909132
valid_h2_mean_x.max_u: 0.684168934822
valid_h2_mean_x.mean_u: 0.353794693947
valid_h2_mean_x.min_u: -0.0889124572277
valid_h2_min_x.max_u: -0.251966834068
valid_h2_min_x.mean_u: -2.23872900009
valid_h2_min_x.min_u: -3.58449077606
valid_h2_range_x.max_u: 9.84970664978
valid_h2_range_x.mean_u: 6.19744825363
valid_h2_range_x.min_u: 1.96390032768
valid_objective: 3.3347492218
valid_y_col_norms_max: 1.02679443359
valid_y_col_norms_mean: 0.416194468737
valid_y_col_norms_min: 0.178296133876
valid_y_max_max_class: 0.869118571281
valid_y_mean_max_class: 0.442211717367
valid_y_min_max_class: 0.119910597801
valid_y_misclass: 0.760190188885
valid_y_nll: 3.51235342026
valid_y_row_norms_max: 0.42383363843
valid_y_row_norms_mean: 0.151956483722
valid_y_row_norms_min: 0.0329562090337
Saving to /disk/scratch/neuroglycerin/models/first_pylearn2_run.pkl...
Saving to /disk/scratch/neuroglycerin/models/first_pylearn2_run.pkl done. Time elapsed: 0.273824 seconds
Time this epoch: 42.388099 seconds
Monitoring step:
Epochs seen: 4
Batches seen: 756
Examples seen: 96768
learning_rate: 0.0500000156462
momentum: 0.66333335638
test_h0_kernel_norms_max: 0.535991132259
test_h0_kernel_norms_mean: 0.171290040016
test_h0_kernel_norms_min: 0.024702694267
test_h0_max_x.max_u: 0.740340292454
test_h0_max_x.mean_u: 0.183671623468
test_h0_max_x.min_u: -0.076398961246
test_h0_mean_x.max_u: -0.0416524931788
test_h0_mean_x.mean_u: -0.35616850853
test_h0_mean_x.min_u: -0.663487792015
test_h0_min_x.max_u: -0.0539095699787
test_h0_min_x.mean_u: -0.46822822094
test_h0_min_x.min_u: -0.921264529228
test_h0_range_x.max_u: 1.5639591217
test_h0_range_x.mean_u: 0.651899695396
test_h0_range_x.min_u: 0.115356437862
test_h1_kernel_norms_max: 0.684282183647
test_h1_kernel_norms_mean: 0.346445083618
test_h1_kernel_norms_min: 0.176242485642
test_h1_max_x.max_u: 3.49669742584
test_h1_max_x.mean_u: 1.88570189476
test_h1_max_x.min_u: 0.168029218912
test_h1_mean_x.max_u: 0.96785825491
test_h1_mean_x.mean_u: -0.0977243930101
test_h1_mean_x.min_u: -1.26161932945
test_h1_min_x.max_u: -0.207417115569
test_h1_min_x.mean_u: -1.45882332325
test_h1_min_x.min_u: -3.07610177994
test_h1_range_x.max_u: 5.17167949677
test_h1_range_x.mean_u: 3.34452581406
test_h1_range_x.min_u: 1.04329109192
test_h2_kernel_norms_max: 0.818537294865
test_h2_kernel_norms_mean: 0.38146084547
test_h2_kernel_norms_min: 0.122520282865
test_h2_max_x.max_u: 6.13806724548
test_h2_max_x.mean_u: 3.65382933617
test_h2_max_x.min_u: 1.94014763832
test_h2_mean_x.max_u: 0.623040258884
test_h2_mean_x.mean_u: 0.21900857985
test_h2_mean_x.min_u: -0.314087599516
test_h2_min_x.max_u: -0.503148674965
test_h2_min_x.mean_u: -2.40146183968
test_h2_min_x.min_u: -3.71244502068
test_h2_range_x.max_u: 8.82101535797
test_h2_range_x.mean_u: 6.05529165268
test_h2_range_x.min_u: 2.67890644073
test_objective: 3.05075836182
test_y_col_norms_max: 1.17799651623
test_y_col_norms_mean: 0.529372990131
test_y_col_norms_min: 0.227641910315
test_y_max_max_class: 0.93388569355
test_y_mean_max_class: 0.516840696335
test_y_min_max_class: 0.122493587434
test_y_misclass: 0.713541567326
test_y_nll: 3.26237106323
test_y_row_norms_max: 0.519736468792
test_y_row_norms_mean: 0.196485057473
test_y_row_norms_min: 0.0358680747449
total_seconds_last_epoch: 47.779548645
training_seconds_this_epoch: 42.3880958557
valid_h0_kernel_norms_max: 0.535991072655
valid_h0_kernel_norms_mean: 0.171290010214
valid_h0_kernel_norms_min: 0.0247026830912
valid_h0_max_x.max_u: 0.740047574043
valid_h0_max_x.mean_u: 0.180688634515
valid_h0_max_x.min_u: -0.078162394464
valid_h0_mean_x.max_u: -0.0417144857347
valid_h0_mean_x.mean_u: -0.357597321272
valid_h0_mean_x.min_u: -0.667439103127
valid_h0_min_x.max_u: -0.0539243780077
valid_h0_min_x.mean_u: -0.467552632093
valid_h0_min_x.min_u: -0.924739778042
valid_h0_range_x.max_u: 1.54138708115
valid_h0_range_x.mean_u: 0.648241400719
valid_h0_range_x.min_u: 0.114996165037
valid_h1_kernel_norms_max: 0.684282183647
valid_h1_kernel_norms_mean: 0.34644523263
valid_h1_kernel_norms_min: 0.176242440939
valid_h1_max_x.max_u: 3.39933252335
valid_h1_max_x.mean_u: 1.83645772934
valid_h1_max_x.min_u: 0.175201371312
valid_h1_mean_x.max_u: 0.961535811424
valid_h1_mean_x.mean_u: -0.108643122017
valid_h1_mean_x.min_u: -1.27830088139
valid_h1_min_x.max_u: -0.206127047539
valid_h1_min_x.mean_u: -1.45465874672
valid_h1_min_x.min_u: -3.06814908981
valid_h1_range_x.max_u: 5.05137586594
valid_h1_range_x.mean_u: 3.29111647606
valid_h1_range_x.min_u: 1.02838253975
valid_h2_kernel_norms_max: 0.818537354469
valid_h2_kernel_norms_mean: 0.381460815668
valid_h2_kernel_norms_min: 0.122520335019
valid_h2_max_x.max_u: 5.98397874832
valid_h2_max_x.mean_u: 3.59713220596
valid_h2_max_x.min_u: 1.89372837543
valid_h2_mean_x.max_u: 0.626972794533
valid_h2_mean_x.mean_u: 0.224481761456
valid_h2_mean_x.min_u: -0.299534231424
valid_h2_min_x.max_u: -0.499290496111
valid_h2_min_x.mean_u: -2.39320921898
valid_h2_min_x.min_u: -3.75206446648
valid_h2_range_x.max_u: 8.58625030518
valid_h2_range_x.mean_u: 5.99034118652
valid_h2_range_x.min_u: 2.63856244087
valid_objective: 3.00071382523
valid_y_col_norms_max: 1.17799651623
valid_y_col_norms_mean: 0.529372990131
valid_y_col_norms_min: 0.227641835809
valid_y_max_max_class: 0.935289204121
valid_y_mean_max_class: 0.517929375172
valid_y_min_max_class: 0.128187343478
valid_y_misclass: 0.712975561619
valid_y_nll: 3.18759393692
valid_y_row_norms_max: 0.519736647606
valid_y_row_norms_mean: 0.196485117078
valid_y_row_norms_min: 0.0358680672944
Saving to /disk/scratch/neuroglycerin/models/first_pylearn2_run.pkl...
Saving to /disk/scratch/neuroglycerin/models/first_pylearn2_run.pkl done. Time elapsed: 0.277758 seconds
Time this epoch: 42.368445 seconds
Monitoring step:
Epochs seen: 5
Batches seen: 945
Examples seen: 120960
learning_rate: 0.0500000156462
momentum: 0.717777788639
test_h0_kernel_norms_max: 0.900000452995
test_h0_kernel_norms_mean: 0.900000214577
test_h0_kernel_norms_min: 0.900000095367
test_h0_max_x.max_u: 1.939e+06
test_h0_max_x.mean_u: 3.320e+05
test_h0_max_x.min_u: 1.990e+04
test_h0_mean_x.max_u: 4.637e+05
test_h0_mean_x.mean_u: 4.917e+04
test_h0_mean_x.min_u: -8.022e+04
test_h0_min_x.max_u: 2208.58837891
test_h0_min_x.mean_u: -9.869e+04
test_h0_min_x.min_u: -1.054e+06
test_h0_range_x.max_u: 2.992e+06
test_h0_range_x.mean_u: 4.306e+05
test_h0_range_x.min_u: 4.431e+04
test_h1_kernel_norms_max: 1.93650281429
test_h1_kernel_norms_mean: 1.9365003109
test_h1_kernel_norms_min: 1.93649756908
test_h1_max_x.max_u: 1.533e+07
test_h1_max_x.mean_u: 4.080e+06
test_h1_max_x.min_u: -5.492e+05
test_h1_mean_x.max_u: 1.188e+07
test_h1_mean_x.mean_u: 6.131e+05
test_h1_mean_x.min_u: -6.223e+06
test_h1_min_x.max_u: 5.085e+06
test_h1_min_x.mean_u: -3.631e+06
test_h1_min_x.min_u: -1.278e+07
test_h1_range_x.max_u: 1.267e+07
test_h1_range_x.mean_u: 7.710e+06
test_h1_range_x.min_u: 3.007e+06
test_h2_kernel_norms_max: 1.93650114536
test_h2_kernel_norms_mean: 1.9365003109
test_h2_kernel_norms_min: 1.93649792671
test_h2_max_x.max_u: 1.391e+08
test_h2_max_x.mean_u: 2.639e+07
test_h2_max_x.min_u: -3.500e+06
test_h2_mean_x.max_u: 5.973e+07
test_h2_mean_x.mean_u: -9.866e+06
test_h2_mean_x.min_u: -4.494e+07
test_h2_min_x.max_u: 9.322e+06
test_h2_min_x.mean_u: -7.069e+07
test_h2_min_x.min_u: -1.410e+08
test_h2_range_x.max_u: 1.527e+08
test_h2_range_x.mean_u: 9.708e+07
test_h2_range_x.min_u: 3.226e+07
test_objective: 1.218e+14
test_y_col_norms_max: 2.399e+05
test_y_col_norms_mean: 2.458e+04
test_y_col_norms_min: 1.99148583412
test_y_max_max_class: 1.00000011921
test_y_mean_max_class: 1.00000011921
test_y_min_max_class: 1.00000011921
test_y_misclass: 0.998698055744
test_y_nll: 1.204e+14
test_y_row_norms_max: 7.442e+04
test_y_row_norms_mean: 1.593e+04
test_y_row_norms_min: 1282.77807617
total_seconds_last_epoch: 47.797203064
training_seconds_this_epoch: 42.3684425354
valid_h0_kernel_norms_max: 0.900000274181
valid_h0_kernel_norms_mean: 0.900000035763
valid_h0_kernel_norms_min: 0.899999976158
valid_h0_max_x.max_u: 1.939e+06
valid_h0_max_x.mean_u: 3.320e+05
valid_h0_max_x.min_u: 1.990e+04
valid_h0_mean_x.max_u: 4.637e+05
valid_h0_mean_x.mean_u: 4.917e+04
valid_h0_mean_x.min_u: -8.022e+04
valid_h0_min_x.max_u: 2208.72729492
valid_h0_min_x.mean_u: -9.869e+04
valid_h0_min_x.min_u: -1.054e+06
valid_h0_range_x.max_u: 2.992e+06
valid_h0_range_x.mean_u: 4.306e+05
valid_h0_range_x.min_u: 4.431e+04
valid_h1_kernel_norms_max: 1.93650209904
valid_h1_kernel_norms_mean: 1.93649935722
valid_h1_kernel_norms_min: 1.93649697304
valid_h1_max_x.max_u: 1.533e+07
valid_h1_max_x.mean_u: 4.080e+06
valid_h1_max_x.min_u: -5.492e+05
valid_h1_mean_x.max_u: 1.188e+07
valid_h1_mean_x.mean_u: 6.131e+05
valid_h1_mean_x.min_u: -6.223e+06
valid_h1_min_x.max_u: 5.085e+06
valid_h1_min_x.mean_u: -3.631e+06
valid_h1_min_x.min_u: -1.278e+07
valid_h1_range_x.max_u: 1.267e+07
valid_h1_range_x.mean_u: 7.710e+06
valid_h1_range_x.min_u: 3.007e+06
valid_h2_kernel_norms_max: 1.93650174141
valid_h2_kernel_norms_mean: 1.93649971485
valid_h2_kernel_norms_min: 1.93649888039
valid_h2_max_x.max_u: 1.391e+08
valid_h2_max_x.mean_u: 2.639e+07
valid_h2_max_x.min_u: -3.500e+06
valid_h2_mean_x.max_u: 5.973e+07
valid_h2_mean_x.mean_u: -9.866e+06
valid_h2_mean_x.min_u: -4.494e+07
valid_h2_min_x.max_u: 9.322e+06
valid_h2_min_x.mean_u: -7.069e+07
valid_h2_min_x.min_u: -1.410e+08
valid_h2_range_x.max_u: 1.527e+08
valid_h2_range_x.mean_u: 9.708e+07
valid_h2_range_x.min_u: 3.226e+07
valid_objective: 1.222e+14
valid_y_col_norms_max: 2.399e+05
valid_y_col_norms_mean: 2.458e+04
valid_y_col_norms_min: 1.99148595333
valid_y_max_max_class: 0.999999821186
valid_y_mean_max_class: 0.999999821186
valid_y_min_max_class: 0.999999821186
valid_y_misclass: 0.998980820179
valid_y_nll: 1.211e+14
valid_y_row_norms_max: 7.442e+04
valid_y_row_norms_mean: 1.593e+04
valid_y_row_norms_min: 1282.77770996
Time this epoch: 42.331841 seconds
Monitoring step:
Epochs seen: 6
Batches seen: 1134
Examples seen: 145152
learning_rate: 0.0500000156462
momentum: 0.772222042084
test_h0_kernel_norms_max: 0.900000452995
test_h0_kernel_norms_mean: 0.900000214577
test_h0_kernel_norms_min: 0.900000095367
test_h0_max_x.max_u: 1.567e+07
test_h0_max_x.mean_u: 4.923e+06
test_h0_max_x.min_u: 8.501e+05
test_h0_mean_x.max_u: 4.578e+06
test_h0_mean_x.mean_u: 5.420e+05
test_h0_mean_x.min_u: -8.016e+04
test_h0_min_x.max_u: -3.367e+05
test_h0_min_x.mean_u: -2.271e+06
test_h0_min_x.min_u: -8.127e+06
test_h0_range_x.max_u: 2.380e+07
test_h0_range_x.mean_u: 7.194e+06
test_h0_range_x.min_u: 1.187e+06
test_h1_kernel_norms_max: 1.93650329113
test_h1_kernel_norms_mean: 1.9365003109
test_h1_kernel_norms_min: 1.93649744987
test_h1_max_x.max_u: 1.658e+08
test_h1_max_x.mean_u: 2.355e+07
test_h1_max_x.min_u: -1.464e+07
test_h1_mean_x.max_u: 1.071e+08
test_h1_mean_x.mean_u: -1.907e+07
test_h1_mean_x.min_u: -6.891e+07
test_h1_min_x.max_u: 5.825e+07
test_h1_min_x.mean_u: -5.711e+07
test_h1_min_x.min_u: -1.192e+08
test_h1_range_x.max_u: 1.322e+08
test_h1_range_x.mean_u: 8.067e+07
test_h1_range_x.min_u: 3.225e+07
test_h2_kernel_norms_max: 1.93650126457
test_h2_kernel_norms_mean: 1.9365003109
test_h2_kernel_norms_min: 1.93649792671
test_h2_max_x.max_u: 2.858e+09
test_h2_max_x.mean_u: 1.880e+09
test_h2_max_x.min_u: 1.499e+09
test_h2_mean_x.max_u: 1.312e+09
test_h2_mean_x.mean_u: 8.897e+08
test_h2_mean_x.min_u: 7.339e+08
test_h2_min_x.max_u: 1.852e+08
test_h2_min_x.mean_u: 1.283e+08
test_h2_min_x.min_u: 7.229e+07
test_h2_range_x.max_u: 2.706e+09
test_h2_range_x.mean_u: 1.752e+09
test_h2_range_x.min_u: 1.387e+09
test_objective: 4.022e+14
test_y_col_norms_max: 5.830e+05
test_y_col_norms_mean: 1.264e+04
test_y_col_norms_min: 3.1719892025
test_y_max_max_class: 1.00000011921
test_y_mean_max_class: 1.00000011921
test_y_min_max_class: 1.00000011921
test_y_misclass: 0.999023616314
test_y_nll: 4.138e+14
test_y_row_norms_max: 6.027e+04
test_y_row_norms_mean: 1.818e+04
test_y_row_norms_min: 1171.07324219
total_seconds_last_epoch: 47.4932518005
training_seconds_this_epoch: 42.3318443298
valid_h0_kernel_norms_max: 0.900000274181
valid_h0_kernel_norms_mean: 0.900000035763
valid_h0_kernel_norms_min: 0.899999976158
valid_h0_max_x.max_u: 1.567e+07
valid_h0_max_x.mean_u: 4.923e+06
valid_h0_max_x.min_u: 8.501e+05
valid_h0_mean_x.max_u: 4.578e+06
valid_h0_mean_x.mean_u: 5.420e+05
valid_h0_mean_x.min_u: -8.016e+04
valid_h0_min_x.max_u: -3.367e+05
valid_h0_min_x.mean_u: -2.271e+06
valid_h0_min_x.min_u: -8.127e+06
valid_h0_range_x.max_u: 2.380e+07
valid_h0_range_x.mean_u: 7.194e+06
valid_h0_range_x.min_u: 1.187e+06
valid_h1_kernel_norms_max: 1.93650245667
valid_h1_kernel_norms_mean: 1.93649971485
valid_h1_kernel_norms_min: 1.9364964962
valid_h1_max_x.max_u: 1.658e+08
valid_h1_max_x.mean_u: 2.355e+07
valid_h1_max_x.min_u: -1.464e+07
valid_h1_mean_x.max_u: 1.071e+08
valid_h1_mean_x.mean_u: -1.907e+07
valid_h1_mean_x.min_u: -6.891e+07
valid_h1_min_x.max_u: 5.825e+07
valid_h1_min_x.mean_u: -5.711e+07
valid_h1_min_x.min_u: -1.192e+08
valid_h1_range_x.max_u: 1.322e+08
valid_h1_range_x.mean_u: 8.067e+07
valid_h1_range_x.min_u: 3.225e+07
valid_h2_kernel_norms_max: 1.93650186062
valid_h2_kernel_norms_mean: 1.93649971485
valid_h2_kernel_norms_min: 1.93649852276
valid_h2_max_x.max_u: 2.858e+09
valid_h2_max_x.mean_u: 1.880e+09
valid_h2_max_x.min_u: 1.499e+09
valid_h2_mean_x.max_u: 1.312e+09
valid_h2_mean_x.mean_u: 8.897e+08
valid_h2_mean_x.min_u: 7.339e+08
valid_h2_min_x.max_u: 1.852e+08
valid_h2_min_x.mean_u: 1.283e+08
valid_h2_min_x.min_u: 7.229e+07
valid_h2_range_x.max_u: 2.706e+09
valid_h2_range_x.mean_u: 1.752e+09
valid_h2_range_x.min_u: 1.387e+09
valid_objective: 3.945e+14
valid_y_col_norms_max: 5.830e+05
valid_y_col_norms_mean: 1.264e+04
valid_y_col_norms_min: 3.1719892025
valid_y_max_max_class: 0.999999821186
valid_y_mean_max_class: 0.999999821186
valid_y_min_max_class: 0.999999821186
valid_y_misclass: 0.99932050705
valid_y_nll: 4.071e+14
valid_y_row_norms_max: 6.027e+04
valid_y_row_norms_mean: 1.818e+04
valid_y_row_norms_min: 1171.07299805
Time this epoch: 42.346430 seconds
Monitoring step:
Epochs seen: 7
Batches seen: 1323
Examples seen: 169344
learning_rate: 0.0500000156462
momentum: 0.826666772366
test_h0_kernel_norms_max: 0.900000452995
test_h0_kernel_norms_mean: 0.900000214577
test_h0_kernel_norms_min: 0.899999499321
test_h0_max_x.max_u: 2.775e+07
test_h0_max_x.mean_u: 7.419e+06
test_h0_max_x.min_u: 5.327e+05
test_h0_mean_x.max_u: 5.297e+06
test_h0_mean_x.mean_u: 5.323e+05
test_h0_mean_x.min_u: -1.295e+06
test_h0_min_x.max_u: -4.760e+05
test_h0_min_x.mean_u: -3.821e+06
test_h0_min_x.min_u: -1.306e+07
test_h0_range_x.max_u: 3.382e+07
test_h0_range_x.mean_u: 1.124e+07
test_h0_range_x.min_u: 1.009e+06
test_h1_kernel_norms_max: 1.93650269508
test_h1_kernel_norms_mean: 1.9365003109
test_h1_kernel_norms_min: 1.9364978075
test_h1_max_x.max_u: 2.397e+08
test_h1_max_x.mean_u: 9.978e+07
test_h1_max_x.min_u: 2.108e+07
test_h1_mean_x.max_u: 1.348e+08
test_h1_mean_x.mean_u: 2.642e+07
test_h1_mean_x.min_u: -5.949e+07
test_h1_min_x.max_u: 1.362e+07
test_h1_min_x.mean_u: -7.169e+07
test_h1_min_x.min_u: -2.300e+08
test_h1_range_x.max_u: 2.599e+08
test_h1_range_x.mean_u: 1.715e+08
test_h1_range_x.min_u: 6.708e+07
test_h2_kernel_norms_max: 1.93650126457
test_h2_kernel_norms_mean: 1.9365003109
test_h2_kernel_norms_min: 1.93649792671
test_h2_max_x.max_u: 2.937e+09
test_h2_max_x.mean_u: 6.699e+08
test_h2_max_x.min_u: -5.126e+07
test_h2_mean_x.max_u: 1.268e+09
test_h2_mean_x.mean_u: 1.429e+08
test_h2_mean_x.min_u: -4.167e+08
test_h2_min_x.max_u: 1.304e+08
test_h2_min_x.mean_u: -3.504e+08
test_h2_min_x.min_u: -1.094e+09
test_h2_range_x.max_u: 2.806e+09
test_h2_range_x.mean_u: 1.020e+09
test_h2_range_x.min_u: 4.722e+08
test_objective: 1.559e+16
test_y_col_norms_max: 1.830e+06
test_y_col_norms_mean: 8.272e+04
test_y_col_norms_min: 418.869384766
test_y_max_max_class: 1.00000011921
test_y_mean_max_class: 1.00000011921
test_y_min_max_class: 1.00000011921
test_y_misclass: 0.999674618244
test_y_nll: 1.471e+16
test_y_row_norms_max: 3.764e+05
test_y_row_norms_mean: 6.468e+04
test_y_row_norms_min: 3372.04663086
total_seconds_last_epoch: 47.4537658691
training_seconds_this_epoch: 42.3464317322
valid_h0_kernel_norms_max: 0.900000274181
valid_h0_kernel_norms_mean: 0.900000035763
valid_h0_kernel_norms_min: 0.899999976158
valid_h0_max_x.max_u: 2.775e+07
valid_h0_max_x.mean_u: 7.419e+06
valid_h0_max_x.min_u: 5.327e+05
valid_h0_mean_x.max_u: 5.297e+06
valid_h0_mean_x.mean_u: 5.323e+05
valid_h0_mean_x.min_u: -1.295e+06
valid_h0_min_x.max_u: -4.760e+05
valid_h0_min_x.mean_u: -3.821e+06
valid_h0_min_x.min_u: -1.306e+07
valid_h0_range_x.max_u: 3.382e+07
valid_h0_range_x.mean_u: 1.124e+07
valid_h0_range_x.min_u: 1.009e+06
valid_h1_kernel_norms_max: 1.93650197983
valid_h1_kernel_norms_mean: 1.93649971485
valid_h1_kernel_norms_min: 1.93649852276
valid_h1_max_x.max_u: 2.397e+08
valid_h1_max_x.mean_u: 9.978e+07
valid_h1_max_x.min_u: 2.108e+07
valid_h1_mean_x.max_u: 1.348e+08
valid_h1_mean_x.mean_u: 2.642e+07
valid_h1_mean_x.min_u: -5.949e+07
valid_h1_min_x.max_u: 1.362e+07
valid_h1_min_x.mean_u: -7.169e+07
valid_h1_min_x.min_u: -2.300e+08
valid_h1_range_x.max_u: 2.599e+08
valid_h1_range_x.mean_u: 1.715e+08
valid_h1_range_x.min_u: 6.708e+07
valid_h2_kernel_norms_max: 1.93650186062
valid_h2_kernel_norms_mean: 1.93649971485
valid_h2_kernel_norms_min: 1.93649852276
valid_h2_max_x.max_u: 2.937e+09
valid_h2_max_x.mean_u: 6.699e+08
valid_h2_max_x.min_u: -5.126e+07
valid_h2_mean_x.max_u: 1.268e+09
valid_h2_mean_x.mean_u: 1.429e+08
valid_h2_mean_x.min_u: -4.167e+08
valid_h2_min_x.max_u: 1.304e+08
valid_h2_min_x.mean_u: -3.504e+08
valid_h2_min_x.min_u: -1.094e+09
valid_h2_range_x.max_u: 2.806e+09
valid_h2_range_x.mean_u: 1.020e+09
valid_h2_range_x.min_u: 4.722e+08
valid_objective: 1.570e+16
valid_y_col_norms_max: 1.830e+06
valid_y_col_norms_mean: 8.272e+04
valid_y_col_norms_min: 418.869598389
valid_y_max_max_class: 0.999999821186
valid_y_mean_max_class: 0.999999821186
valid_y_min_max_class: 0.999999821186
valid_y_misclass: 0.999660134315
valid_y_nll: 1.470e+16
valid_y_row_norms_max: 3.764e+05
valid_y_row_norms_mean: 6.468e+04
valid_y_row_norms_min: 3372.04516602
Time this epoch: 42.350535 seconds
Monitoring step:
Epochs seen: 8
Batches seen: 1512
Examples seen: 193536
learning_rate: 0.0500000156462
momentum: 0.881110966206
test_h0_kernel_norms_max: 0.900000452995
test_h0_kernel_norms_mean: 0.900000214577
test_h0_kernel_norms_min: 0.900000095367
test_h0_max_x.max_u: 4.568e+06
test_h0_max_x.mean_u: 1.912e+06
test_h0_max_x.min_u: 2.465e+05
test_h0_mean_x.max_u: 1.773e+05
test_h0_mean_x.mean_u: -1321.26171875
test_h0_mean_x.min_u: -1.968e+05
test_h0_min_x.max_u: -2.356e+05
test_h0_min_x.mean_u: -1.992e+06
test_h0_min_x.min_u: -5.238e+06
test_h0_range_x.max_u: 8.701e+06
test_h0_range_x.mean_u: 3.904e+06
test_h0_range_x.min_u: 4.821e+05
test_h1_kernel_norms_max: 1.93650317192
test_h1_kernel_norms_mean: 1.9365003109
test_h1_kernel_norms_min: 1.93649744987
test_h1_max_x.max_u: 3.285e+07
test_h1_max_x.mean_u: 1.631e+07
test_h1_max_x.min_u: 4.165e+06
test_h1_mean_x.max_u: 7.954e+06
test_h1_mean_x.mean_u: 2.687e+06
test_h1_mean_x.min_u: -5.522e+05
test_h1_min_x.max_u: -5.373e+05
test_h1_min_x.mean_u: -3.747e+06
test_h1_min_x.min_u: -7.108e+06
test_h1_range_x.max_u: 3.514e+07
test_h1_range_x.mean_u: 2.006e+07
test_h1_range_x.min_u: 9.676e+06
test_h2_kernel_norms_max: 1.93650126457
test_h2_kernel_norms_mean: 1.9365003109
test_h2_kernel_norms_min: 1.93649792671
test_h2_max_x.max_u: 3.863e+08
test_h2_max_x.mean_u: 1.755e+08
test_h2_max_x.min_u: 1.072e+07
test_h2_mean_x.max_u: 1.219e+08
test_h2_mean_x.mean_u: 4.366e+07
test_h2_mean_x.min_u: -3.361e+07
test_h2_min_x.max_u: 1.804e+07
test_h2_min_x.mean_u: -4.967e+07
test_h2_min_x.min_u: -2.115e+08
test_h2_range_x.max_u: 3.840e+08
test_h2_range_x.mean_u: 2.251e+08
test_h2_range_x.min_u: 8.961e+07
test_objective: 4.762e+12
test_y_col_norms_max: 2.928e+04
test_y_col_norms_mean: 1871.82702637
test_y_col_norms_min: 39.0379905701
test_y_max_max_class: 1.00000011921
test_y_mean_max_class: 1.00000011921
test_y_min_max_class: 1.00000011921
test_y_misclass: 0.999023616314
test_y_nll: 4.159e+12
test_y_row_norms_max: 4949.11083984
test_y_row_norms_mean: 1152.45812988
test_y_row_norms_min: 97.2359542847
total_seconds_last_epoch: 47.4625778198
training_seconds_this_epoch: 42.3505477905
valid_h0_kernel_norms_max: 0.900000274181
valid_h0_kernel_norms_mean: 0.900000035763
valid_h0_kernel_norms_min: 0.899999976158
valid_h0_max_x.max_u: 4.568e+06
valid_h0_max_x.mean_u: 1.912e+06
valid_h0_max_x.min_u: 2.465e+05
valid_h0_mean_x.max_u: 1.773e+05
valid_h0_mean_x.mean_u: -1321.25866699
valid_h0_mean_x.min_u: -1.968e+05
valid_h0_min_x.max_u: -2.356e+05
valid_h0_min_x.mean_u: -1.992e+06
valid_h0_min_x.min_u: -5.238e+06
valid_h0_range_x.max_u: 8.701e+06
valid_h0_range_x.mean_u: 3.904e+06
valid_h0_range_x.min_u: 4.821e+05
valid_h1_kernel_norms_max: 1.93650245667
valid_h1_kernel_norms_mean: 1.93649971485
valid_h1_kernel_norms_min: 1.93649661541
valid_h1_max_x.max_u: 3.285e+07
valid_h1_max_x.mean_u: 1.631e+07
valid_h1_max_x.min_u: 4.165e+06
valid_h1_mean_x.max_u: 7.954e+06
valid_h1_mean_x.mean_u: 2.687e+06
valid_h1_mean_x.min_u: -5.522e+05
valid_h1_min_x.max_u: -5.373e+05
valid_h1_min_x.mean_u: -3.747e+06
valid_h1_min_x.min_u: -7.108e+06
valid_h1_range_x.max_u: 3.514e+07
valid_h1_range_x.mean_u: 2.006e+07
valid_h1_range_x.min_u: 9.676e+06
valid_h2_kernel_norms_max: 1.93650174141
valid_h2_kernel_norms_mean: 1.93649971485
valid_h2_kernel_norms_min: 1.93649852276
valid_h2_max_x.max_u: 3.863e+08
valid_h2_max_x.mean_u: 1.755e+08
valid_h2_max_x.min_u: 1.072e+07
valid_h2_mean_x.max_u: 1.219e+08
valid_h2_mean_x.mean_u: 4.366e+07
valid_h2_mean_x.min_u: -3.361e+07
valid_h2_min_x.max_u: 1.804e+07
valid_h2_min_x.mean_u: -4.967e+07
valid_h2_min_x.min_u: -2.115e+08
valid_h2_range_x.max_u: 3.840e+08
valid_h2_range_x.mean_u: 2.251e+08
valid_h2_range_x.min_u: 8.961e+07
valid_objective: 4.795e+12
valid_y_col_norms_max: 2.928e+04
valid_y_col_norms_mean: 1871.82702637
valid_y_col_norms_min: 39.0379829407
valid_y_max_max_class: 0.999999821186
valid_y_mean_max_class: 0.999999821186
valid_y_min_max_class: 0.999999821186
valid_y_misclass: 0.998980820179
valid_y_nll: 4.201e+12
valid_y_row_norms_max: 4949.11132812
valid_y_row_norms_mean: 1152.45788574
valid_y_row_norms_min: 97.2359771729
Time this epoch: 42.355780 seconds
Monitoring step:
Epochs seen: 9
Batches seen: 1701
Examples seen: 217728
learning_rate: 0.0500000156462
momentum: 0.935555815697
test_h0_kernel_norms_max: 0.900000452995
test_h0_kernel_norms_mean: 0.900000214577
test_h0_kernel_norms_min: 0.900000095367
test_h0_max_x.max_u: 3.969e+06
test_h0_max_x.mean_u: 1.217e+06
test_h0_max_x.min_u: 2.372e+05
test_h0_mean_x.max_u: 2.018e+05
test_h0_mean_x.mean_u: -7.563e+04
test_h0_mean_x.min_u: -4.003e+05
test_h0_min_x.max_u: -2.041e+05
test_h0_min_x.mean_u: -1.521e+06
test_h0_min_x.min_u: -4.723e+06
test_h0_range_x.max_u: 7.100e+06
test_h0_range_x.mean_u: 2.739e+06
test_h0_range_x.min_u: 4.413e+05
test_h1_kernel_norms_max: 1.93650317192
test_h1_kernel_norms_mean: 1.9365003109
test_h1_kernel_norms_min: 1.93649756908
test_h1_max_x.max_u: 5.036e+07
test_h1_max_x.mean_u: 1.805e+07
test_h1_max_x.min_u: 1.436e+06
test_h1_mean_x.max_u: 1.965e+07
test_h1_mean_x.mean_u: 6.122e+06
test_h1_mean_x.min_u: -6.168e+06
test_h1_min_x.max_u: 6.460e+06
test_h1_min_x.mean_u: -4.637e+06
test_h1_min_x.min_u: -2.076e+07
test_h1_range_x.max_u: 4.674e+07
test_h1_range_x.mean_u: 2.269e+07
test_h1_range_x.min_u: 1.371e+07
test_h2_kernel_norms_max: 1.93650126457
test_h2_kernel_norms_mean: 1.9365003109
test_h2_kernel_norms_min: 1.93649792671
test_h2_max_x.max_u: 3.834e+08
test_h2_max_x.mean_u: 1.050e+08
test_h2_max_x.min_u: -5.334e+06
test_h2_mean_x.max_u: 1.839e+08
test_h2_mean_x.mean_u: 3.863e+07
test_h2_mean_x.min_u: -7.629e+07
test_h2_min_x.max_u: 2.217e+07
test_h2_min_x.mean_u: -2.280e+07
test_h2_min_x.min_u: -2.451e+08
test_h2_range_x.max_u: 3.612e+08
test_h2_range_x.mean_u: 1.278e+08
test_h2_range_x.min_u: 3.363e+07
test_objective: 8.766e+13
test_y_col_norms_max: 1.291e+05
test_y_col_norms_mean: 2.791e+04
test_y_col_norms_min: 1640.64343262
test_y_max_max_class: 1.00000011921
test_y_mean_max_class: 1.00000011921
test_y_min_max_class: 1.00000011921
test_y_misclass: 0.936849117279
test_y_nll: 8.193e+13
test_y_row_norms_max: 2.961e+04
test_y_row_norms_mean: 1.109e+04
test_y_row_norms_min: 1250.84155273
total_seconds_last_epoch: 47.4680099487
training_seconds_this_epoch: 42.3557777405
valid_h0_kernel_norms_max: 0.900000274181
valid_h0_kernel_norms_mean: 0.900000035763
valid_h0_kernel_norms_min: 0.900000035763
valid_h0_max_x.max_u: 3.969e+06
valid_h0_max_x.mean_u: 1.217e+06
valid_h0_max_x.min_u: 2.372e+05
valid_h0_mean_x.max_u: 2.018e+05
valid_h0_mean_x.mean_u: -7.563e+04
valid_h0_mean_x.min_u: -4.003e+05
valid_h0_min_x.max_u: -2.041e+05
valid_h0_min_x.mean_u: -1.521e+06
valid_h0_min_x.min_u: -4.723e+06
valid_h0_range_x.max_u: 7.100e+06
valid_h0_range_x.mean_u: 2.739e+06
valid_h0_range_x.min_u: 4.413e+05
valid_h1_kernel_norms_max: 1.93650245667
valid_h1_kernel_norms_mean: 1.93649971485
valid_h1_kernel_norms_min: 1.93649697304
valid_h1_max_x.max_u: 5.036e+07
valid_h1_max_x.mean_u: 1.805e+07
valid_h1_max_x.min_u: 1.436e+06
valid_h1_mean_x.max_u: 1.965e+07
valid_h1_mean_x.mean_u: 6.122e+06
valid_h1_mean_x.min_u: -6.168e+06
valid_h1_min_x.max_u: 6.460e+06
valid_h1_min_x.mean_u: -4.637e+06
valid_h1_min_x.min_u: -2.076e+07
valid_h1_range_x.max_u: 4.674e+07
valid_h1_range_x.mean_u: 2.269e+07
valid_h1_range_x.min_u: 1.371e+07
valid_h2_kernel_norms_max: 1.93650174141
valid_h2_kernel_norms_mean: 1.93649971485
valid_h2_kernel_norms_min: 1.93649888039
valid_h2_max_x.max_u: 3.834e+08
valid_h2_max_x.mean_u: 1.050e+08
valid_h2_max_x.min_u: -5.334e+06
valid_h2_mean_x.max_u: 1.839e+08
valid_h2_mean_x.mean_u: 3.863e+07
valid_h2_mean_x.min_u: -7.629e+07
valid_h2_min_x.max_u: 2.217e+07
valid_h2_min_x.mean_u: -2.280e+07
valid_h2_min_x.min_u: -2.451e+08
valid_h2_range_x.max_u: 3.612e+08
valid_h2_range_x.mean_u: 1.278e+08
valid_h2_range_x.min_u: 3.363e+07
valid_objective: 8.745e+13
valid_y_col_norms_max: 1.291e+05
valid_y_col_norms_mean: 2.791e+04
valid_y_col_norms_min: 1640.64331055
valid_y_max_max_class: 0.999999821186
valid_y_mean_max_class: 0.999999821186
valid_y_min_max_class: 0.999999821186
valid_y_misclass: 0.934442818165
valid_y_nll: 8.185e+13
valid_y_row_norms_max: 2.961e+04
valid_y_row_norms_mean: 1.109e+04
valid_y_row_norms_min: 1250.84130859
Content source: Neuroglycerin/neukrill-net-work
Similar notebooks: